Cultural Heritage in CLEF (CHiC) 2013 - Multilingual Task Overview

The Cultural Heritage in CLEF 2013 multilingual task comprised two sub-tasks: multilingual ad-hoc retrieval and semantic enrichment. The multilingual ad-hoc retrieval sub-task evaluated retrieval experiments in 13 languages (Dutch, English, German, Greek, Finnish, French, Hungarian, Italian; Norwegian, Polish, Slovenian, Spanish, Swedish). More than 140,000 documents were assessed for relevance on a tertiary scale. The ad-hoc task had 7 participants submitting 30 multilingual and 41 monolingual runs. The semantic enrichment task evaluated monolingual and multilingual semantic enrichments (suggestions based on a query) in the same 13 languages. Two participants submitted 10 runs. Results indicated that different languages contribute differently to the overall retrieval effectiveness, probably dependent on collection size. Experiments showed that using more or all of the provided languages usually increases retrieval effectiveness, but not always. For a multilingual task of this scale (13 languages), more participants are necessary in order to provide enough variations in runs to allow for comparative analyses.